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ABSTRACT 


A  search  is  conducted  for  a  target  moving  in  discrete  time  among  a  finite  number  of  cells  according  to 
a  known  Markov  process.  The  searcher  must  choose  one  cell  in  which  to  search  in  each  time  period.  The  set 
of  cells  available  for  search  depends  upon  the  cell  chosen  in  the  last  time  period.  The  problem  is  to  find  a 
search  path,  i.e.,  a  sequence  of  search  cells,  that  maximizes  the  probability  of  detecting  the  target  in  a  fixed 
number  of  time  periods.  Closely  following  earlier  work  by  Theodor  Stewart,  a  branch-and-bound  procedure 
is  developed  which  finds  optimal  search  paths.  This  procedure  is  tested  and  appears  to  be  more  efficient 
than  existing  dynamic  programming  solution  methods. 
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A  searcher  and  target  move  among  a  finite  set  of  cells  C  —  1, 2, ...,  N  in  discrete  time.  At  the  beginning 
of  each  time  period,  one  cell  is  searched.  If  the  target  is  in  the  selected  cell  i,  it  is  detected  with  probability 
<?,-.  If  the  target  is  not  in  the  cell  searched,  it  can  not  be  detected  during  the  current  time  period.  After 
each  search,  a  target  in  cell  j  moves  to  cell  k  with  probability  pjk.  The  target  transition  matrix,  P  =  [pjk] 
is  known  to  the  searcher.  The  searcher's  path  is  constrained  in  that  if  the  searcher  is  currently  in  cell  i,  the 
next  search  cell  must  be  selected  from  a  set  of  neighboring  cells  C,.  (The  set  C{  is  also  those  cells  from  which 
i  can  be  reached.)  The  object  of  the  search  is  to  minimize  the  probability  of  not  detecting  the  target  in  T 
searches. 

1.   Background 

The  path  constrained  search  problem,  described  above,  is  a  difficult  one  to  solve  efficiently.  Trummel  and 
Weisinger  [1986]  showed  that  the  path  constrained  search  problem  with  a  stationary  target  is  NP-complete. 
The  moving  target  problem,  which  is  a  generalization  of  the  stationary  target  problem,  is  easily  shown  to 
be  at  least  as  difficult. 

The  only  optimal  solution  technique  mentioned  in  the  literature  for  this  problem  has  been  the  dynamic 
programming  procedure  of  Eagle  [1984a].  Although  this  method  can  solve  problems  much  more  quickly  than 
can  total  enumeration,  it  can  require  a  large  amount  of  computer  storage  as  problem  size  increases. 

There  have  been  several  approximate  solution  procedures  suggested  for  this  problem.  The  first  such 
method  proposed  was  a  branch-and-bound  method  by  Stewart  [1979]  and  [1980].  Stewart's  bounds  were 
obtained  by  solving  an  integer  problem  without  path  constraints.  The  solution  to  this  simpler  problem  was 
obtained  using  a  discrete  version  of  the  moving  target  search  algorithm  given  by  Brown  [1980].  However, 
Brown's  algorithm  does  not  necessarily  give  optimal  solutions  when  search  effort  is  discrete,  so  these  "bounds" 
are  only  approximate  and  can  result  in  an  optimal  branch  of  the  enumeration  tree  being  mistakenly  fathomed. 
Nonetheless,  Stewart's  computational  experience  with  1-dimensional  search  problems  indicates  that  the 
method  can  perform  well. 

Another  approximate  procedure  was  given  by  Eagle  [1984b].  This  dynamic  programming  method  uses 
a  moving  or  "rolling"  time  horizon  that  greatly  reduces  computer  storage  requirements.  It  was  used  to 
approximately  solve  a  small  2-dimensional  problem  (3  by  3  search  grid)  for  40  time  periods.  This  procedure 
generalizes  myopic  search  by  selecting  the  next  cell  to  be  searched  under  the  assumption  that  the  search 
ends  m  time  periods  in  the  future.  For  myopic  search,  m  is  1.  For  small  enough  m,  this  procedure  can  be 
implemented  on  a  microcomputer. 
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Reported  here  is  an  optimal  solution  method  which  appears  to  perform  more  efficiently  than  the  existing 
optimal  dynamic  programming  procedures.  This  work  is  a  direct  continuation  of  that  of  Stewart  [1979].  The 
approximate  bound  in  Stewart's  branch-and-bound  procedure  is  replaced  with  a  true  lower  bound,  thus 
guaranteeing  optimal  solutions.  The  true  bound  is  obtained  by  relaxing  the  integer  problem  in  a  manner 
first  suggested  by  Stewart  [1979].  The  branch-and-bound  procedure  is  made  computationally  feasible  by 
special  structure  in  the  relaxed  problem  which  allows  for  its  efficient  solution. 

2.  The  Relaxed  Problem 

The  procedure  presented  here  depends  critically  on  the  efficient  solution  of  a  convex  nonlinear  program, 
here  called  the  relaxed  problem  or  RP.  The  relaxed  problem,  introduced  by  Stewart  [1979],  allows  infinite 
divisibility  of  search  effort  over  a  subset  of  the  search  cells,  while  choosing  this  subset  properly  to  maintain 
a  representation  of  path  constraints.  Following  Stewart,  let  x(i,  j,t)  be  the  amount  of  search  effort  that  is 
redistributed  from  cell  i  in  time  period  t  to  cell  j  in  time  period  t  +  1.  And  let  X(i,t)  be  the  total  search 
effort  in  cell  i  at  time  t.  Then, 

X(i,t)  =  ]T  z(k,i,t-  1),         e  =  l,2,...fT    and     i=l,...,N. 

The  searcher's  initial  search  effort  distribution  is  X(i,  0),  and  is  assumed  to  be  specified.  For  a  single  searcher 
(our  assumption  here),  we  additionally  require  X(i,0)  =  1  if  i  is  the  searcher's  starting  cell,  and  X{i,  0)  =  0 
otherwise. 

An  exponential  detection  function  is  assumed  for  RP.  That  is,  if  the  search  effort  in  cell  i  at  time  t  is 
X(i,t),  then  the  probability  of  detecting  the  target  during  that  search  is  1  —  exp(— or,^(i,f)).  The  term  orj 
is  selected  so  that  1  —  exp(— a,)  =  <7,-.  This  insures  that  integer  solutions  for  RP  have  the  correct  objective 
function  values.  To  keep  qtj  finite,  we  require  q,  <  1.  Additionally  we  let  u>  =  (u/(l),u/(2), . . .  ,w(T))  be 
a  sample  target  track,  and  pw  be  the  probability  that  the  target  follows  that  path.  Finally,  the  set  of  all 
possible  target  paths  is  Q. 

The  optimal  search  flows  x(i,j,t)  for  RP  are  obtained  by  solving  the  following  nonlinear  program: 

T 


minimize  ^  p(w)exp|- ^aw(<)     ^    x(i,u(t),t  -  1)| 

subject  to: 


wen  t=i  ''ec^) 


X(i,0)-  £x(i,j,0)  =  0,         i=l,...,N 

J^z{i,jtt-1)-  J^  x(i,M)  =  0,         }  =  1,...,N     and     *  =  1,...,T-1 

i€Cj  k€Cj 

x(i,j,t)>0,         i,j  =  l,...,N     and     *  =  0,...,T-1 

As  Stewart  observed,  the  objective  function  for  RP  is  convex,  and  the  constraints  are  those  of  a  network. 
The  integer  problem  (IP)  is  identical  to  RP  except  that  each  x(i,j,t)  must  be  either  1  or  0. 

When  the  target  motion  is  Markovian  with  transition  matrix  P  =  [pij],  the  objective  function  of  RP 
can  be  written 

minimize     tt(1)P1P2  . . .  PTl,  (1) 

where  7r(l)  is  a  row  vector  of  the  target's  initial  distribution  over  the  search  cells  C  (assumed  to  be  known), 
1  is  a  column  vector  of  ones,  and  Pt  is  the  target  transition  matrix  P  with  each  row  i  multiplied  by 
exp(— atiX(i,t)).  The  matrix  Pt  is  substochastic  and  updates  the  (defective)  target  distribution  for  the 
search  in  time  period  t  and  the  target  transition  into  time  period  t  +  1. 


As  Brown  [1980]  pointed  out,  the  objective  function  (1)  can  be  written  by  conditioning  on  the  target's 
cell  in  time  period  t.  Specifically, 

N 

^r(;\Oexp{-^ajx(i,.M-l)}s(;,0,  (2) 

where  r(j,t)  is  the  probability  of  the  target  reaching  cell  j  in  time  period  t  without  being  detected  by  searches 
in  time  periods  1,2,...,*  —  1;  and  s(j,t)  is  the  probability  of  a  target  which  starts  in  cell  j  in  time  period  t 
surviving  the  searches  in  time  periods  t  +  1,  ...,T.  Both  r(-,t)  and  s(-,t)  are  easily  calculated  from  r(-,t  —  1) 
and  s(-,t  +  1),  respectively.  The  recursions  begin  with  r(i,  1)  =  7Tj(l)  and  s(i,T)  =  1,  for  all  cells  i. 

The  importance  of  expressing  the  objective  function  as  in  (2)  is  that  it  allows  a  convenient  calculation 
of  the  gradient.  If  /(x)  is  the  objective  function,  then  elements  of  the  gradient  V/(x)  are 

—f^L  =  -Qjr(j,t  +  l)exV{-  J2  <xjX(k,j,t)}s(j,t  +  l). 

Note  that  all  gradient  elements  are  nonpositive,  since  increasing  the  search  flow  along  any  arc  can  not  increase 
the  probability  of  missing  the  target. 

3.  Solving  the  Relaxed  Problem 

Stewart  solves  RP  essentially  as  suggested  by  Frank  and  Wolfe  [1956].  That  is,  starting  with  a  set 
of  feasible  search  flows  xi,  the  gradient  V/(xi)  is  computed.  Then  x[  is  found  by  minimizing  the  linear 
constraint  V/(xi)x  subject  to  the  network  constraints  of  RP.  An  improved,  feasible  set  of  flows  X2  is  then 
found  by  searching  along  the  line  segment  [xlt  x'j].  The  process  continues,  generating  an  improving  sequence 
of  flows  xi,  X2,  X3, . . .  until  a  stopping  criterion  is  met. 

Stewart's  solution  procedure  for  RP  is  particularly  effective  for  the  search  problem  presented  here  for 
two  reasons: 

1.  Since  this  is  a  single-searcher  problem,  arc  capacity  constraints  (allowed  by  Stewart)  are  not  neces- 
sary. Also  a  single  initial  search  cell  is  specified,  rather  than  a  search  effort  distribution  over  several  cells. 
Consequently,  the  linear  subproblem  reduces  to  an  acyclic,  shortest  path  problem — perhaps  the  simplest  of 
all  nontrivial  linear  programs  to  solve.  The  solution  can  be  obtained  by  dynamic  programming  in  polyno- 
mial time  (each  arc  is  examined  only  once)  and  does  not  require  a  general  network  solver.  We  note  that  the 
network  is  acyclic  because  each  arc  connects  a  time  t  node  to  a  time  t  +  1  node. 

2.  At  each  iteration,  the  Frank- Wolfe  method  produces  a  lower  bound  for  the  optimal  RP  objective 
function  value.  In  particular,  if  xn  is  the  current  feasible  solution  to  RP,  and  x*  is  the  optimal  solution, 
then  by  the  convexity  of /(x), 

/(x*)>/(xn)  +  V/(x„)(x*-xn)<. 

And  since  x^,  minimizes  V/(xi)x  over  all  feasible  flows  x, 

/(x*)>/(xn)4-V/(xn)(x;-xn)'. 

For  convex  programs  with  bounded  feasible  regions  (as  we  have  here),  Frank  and  Wolfe  [1956]  showed  that 
this  lower  bound  for  /(x*)  becomes  exact  as  xn  approaches  x*.  The  bound  is  important  since  a  lower  bound 
to  IP  (which  is  provided  by  a  lower  bound  to  RP)  is  required  for  the  branch-and-bound  solution  of  IP. 


4.  Solving  the  Integer  Problem 

Stewart  [1979]  and  [1980]  gave  a  branch-and-bound  algorithm  for  IP.  The  only  change  suggested  here  is 
the  use  of  RP  for  fathoming.  There  are,  however,  at  least  two  important  implementation  issues.  First,  when 
using  Stewart's  procedure  to  solve  a  particular  instance  of  RP,  the  Frank- Wolfe  steps  need  continue  only  until 
branching  or  fathoming  criteria  are  met.  In  particular,  let  PBEST  be  the  branch-and-bound  procedure's 
current  best  (i.e.,  smallest)  objective  function  value  for  an  integer  solution.  Then  during  the  solution  of  RP, 

1.  If  the  lower  bound  to  RP  (/(xn)  +  V/(xn)(x^  —  xn)')  becomes  greater  than  PBEST,  the  current 
branch  is  fathomed. 

2.  If  the  upper  bound  to  RP  (/(x„))  becomes  smaller  than  PBEST,  then  fathoming  is  impossible,  and 
the  procedure  must  branch  one  time  period  deeper  into  the  enumeration  tree. 

3.  If  the  difference  between  the  current  RP  upper  and  lower  bounds  (V/(xn)(x'„  —  xn)£)  becomes  less 
than  some  specified  e  >  0,  then  the  procedure  branches. 

A  second  observation  is  that  each  x^,  which  is  an  optimal  solution  to  a  linear  subproblem,  is  an  extreme 
point  of  the  feasible  region  for  RP.  Each  x'n  is  thus  integer-valued  and  a  feasible  solution  to  IP.  These 
solutions  can  be  used  to  obtain  an  initial,  integer  starting  solution  and  to  continually  update  PBEST  in  the 
branch-and-bound  procedure. 

6.   Computational  Results 

In  Eagle  [1984a],  a  9-cell,  10-time  period  problem  was  reported  solved  to  optimality  using  dynamic 
programming,  and  requiring  19.3  CPU  minutes  on  a  mainframe  computer.  This  same  problem  required  2.5 
CPU  minutes  using  the  branch-and-bound  procedure  presented  here.  (The  comparison  is  not  entirely  fair 
since  the  dynamic  programming  solution  was,  essentially,  for  all  possible  initial  target  distributions.  Once  an 
optimal  solution  was  obtained  for  any  initial  target  distribution,  only  an  additional  1.5  CPU  seconds  were 
required  to  obtain  an  optimal  solution  for  any  other  initial  target  distribution.)  A  few  larger  problems  have 
been  attempted.  A  25-cell,  10-time  period  problem  required  3.15  CPU  minutes;  and  a  49-cell,  10-time  period 
problem  required  4.7  CPU  minutes.  All  computations  were  done  in  Fortran  77  on  the  Naval  Postgraduate 
School  IBM  3033  mainframe  computer. 
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